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ABSTRACT 

This paper investigates the role of cohesion in oral 
proficiency testing. This study analyzed the cohesion of oral reports 
given by 16 Cypriot high school students studying English as a 
foreign language (EFL) using a modification of M. A* K. Halliday and 
R. Hasan's rating scale, It then compared the objective cohesion 
ratings with impressionistic grades of cohesion assigned by three 
experienced raters. Although the results of the study were not 
conclusive, it was found that the only objective cohesive device that 
correlated with raters' grades for cohesion was the use of 
referential pronouns. The strengths and weaknesses of the objective 
rating scale are discussed. Seven appendixes reproduce models of 
communicative competence, test format, objective rating scales, data 
summaries and correlation matrixes. (MDM) 
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Introduction 

This paper investigates the role of cohesion in oral proficiency testing. The cohesion of 
sixteen oral reports was analyzed and compared with an impressionistic grade of cohesion 
assigned to the sixteen oral reports by three raters. The aim of the study is to see whether there 
is any correlation between the nature and frequency of occurrence of the various kinds of 
cohesive devices in each oral report and the average grade for cohesion that was given to each 
oral report by the three raters. The ultimate goal of this study is to check the validity of rating 
scales used in oral proficiency testing. 
Communicative Competence 

In the early seventies the foreign language teaching profession witnessed a shift in the 
methods of language teaching from the audiolingual method to a more integrated one. The new 
wave of interest, which is known as the communicative competence movement has been 
characterized by the fact that language teachers and researchers have recognized that the mere 
knowledge of grammatical structures and vocabulary are not sufficient for meaningful, 
functional and interactive communication. Rather, the ability to successfully communicate in a 
foreign language depends upon a number of factors which interact with each other. However, 
even though it is broadly accepted that communicative competence goes beyond linguistic 
competence, there is no consensus as to what the components of the construct are. 

There are various models of communicative competence among which the best known 
is the one proposed by Canale and Swain 1 980 and Swain 1 983. In their model Canale and 
Swain include four components, the first two focusing on the use of the linguistic system itself. 
Grammatical competence, component number one, encompasses "knowledge of lexical items 
and the rules of morphology, syntax, sentence grammar, semantics and phonology 1 (Canale and 
Swain. 1980: 29). Discourse competence, the second component, is the ability to connect 
sentences in order to form a meaningful unit made up of a number of utterances. What Canale 
and Swain are referring to in discourse competence is also known as cohesion and coherence. 
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The last two compcments pertain to the functional aspects of communication. Sociolinguistic 
competence, component three, is the knowledg e of the sociocultural rules of language and 
discourse. The fourth component, strategic competence, is described by Canale and Swain 
(1980: 30): "as the verbal and nonverbal communication strategies that may be called into action 
to compensate for breakdowns in communication due to performance variables or insufficient 
competence." 

Another model of communicative competence which was proposed by Bachman 1987 
bears many similarities to that of Canale and Swain (Appendix #1 ). Still there are significant 
differences between the two models, the most notable one concerning the role of strategic 
competence. Bachman acknowledges that strategic competence influences language 
performance but does not consider "strategic competence as solely an aspect of language 
competence" (Bachman 1990: 106). In his opinion, strategic competence is a more general 
ability we employ in both verbal and nonverbal tasks. In Bachman's model, strategic 
competence is included as a completely separate element. Bachman's organizational 
competence corresponds to Canale and Swain's grammatical and discourse competence. Also, 
Bachman expands Canale and Swain's notion of sociolinguistic competence to have wider 
connotations as a major element of pragmatic competence. 
Cohesion 

As seen in the models described above cohesion is an important element of 
communicative competence. Cohesion refers to surface structure linguistic means by which 
texture is created (Halliday and Hasan 1976: 4). Texture is a property of a language unit bigger 
than a sentence as opposed to a series of unrelated sentences. A text is a unit of language in use. 
a semantic unit. Therefore, when we are dealing with the cohesion of a text we are investigating 
the linguistic means which enable a text to function as a single meaningful unit. 
The Data 

This paper focuses on the role of cohesion in oral proficiency. More specifically, it looks 
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at the role of the three major categories of cohesive devices: lexical, referential and conjunction. 
Sixteen oral reports produced by an equivalent number of EFL students in Cyprus constitute the 
data for this study. The students were enrolled in the last year of study at both urban and rural 
high schools. By that time the students had had eight years of instruction in English as a foreign 
language.. The reports are part of a multi-component test of oral proficiency. ( Appendix #2) 
The other components were a group discussion, an oral interview and a role play. The students 
were given the choice of two topics to discuss and they were told that after the discussion each 
of them would have to report on the outcome of the discussion. The format of the test battery 
reflects research findings (Shohamy, 1986) indicating that different speech interactions produce 
different kinds of speech samples. For this study all analyzed reports were responses to a prompt 
on tourism in Cyprus (Appendix ??3). 

The rating scale chosen for the grading of the test battery is an adaptation of the one 
proposed by Bachman and Palmer 1983. This scale was chosen because it is based on a theory 
of language, namely communicative competence (Appendix #4) As mentioned already there 
have been two slight modifications to the scale. First, the students were not given a grade for the 
use of cultural references. Since we are dealing with an EFL and not an ESL setting, students 
should not be expected to be extensively familiar with the second language culture or at least 
this kind of knowledge or familiarity should not be subject to evaluation. Needless to say, 
American and British pop culture have become widespread among Cypriot youth. 

The second change relates to register. As can be seen by the scale a new description for 
register was introduced for each battery component. 



Speaker chose 1 



inappropriate register 



2 



appropriate register but 
used inconsistently 



3 



appropriate register and 
used consistently 



A 
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In the original scale students were graded for the evidence and control of both formal and 
informal registers. In the adopted scale we looked at whether thu speaker chose the appropriate 
register for each task and used it consistently or not. 
Procedure 

The cohesion of each oral report was'analyzed using the model proposed by Halliday and 
Hasan (1976) (Appendix #5). The model for the analysis of cohesion operates according to the 
following principles. The basic concept utilized in analyzing the cohesion of a text is that of a 
tie. A tie refers to the presence of a pair of cohesively related items. Therefore, a tie is a 
relational concept and also a directional one. The relation, however, is asymmetric. We 
differentiate between anaphoric and cataphoric relations. In the former case the presupposed 
element precedes the cohesive device whereas in the latter, the presupposed element follows the 
cohesive device. Anaphoric devices are by far more frequent in English. Another dichotomy in 
devices investigated is that of the endophoric/exophoric tie. An endophoric tie is a tie whose 
elements of the cohesive pair lie inside the text itself An exophoric tie is a tie formed with one 
element of the pair inside the text and another outside the text, both within the context of the 
situation. 

Another principle in the analysis of cohesion is that any sentence can have more than one 
tie in it. Additionally, ties can be classified into immediate* mediate and remote. Immediate 
ties are those in which the cohesive element is located in the adjacent sentence in which the 
presupposed item is embedded. A mediate tie is one in which the cohesive device is related to 
an item in a preceding sentence (but not in the immediately preceding one) through an item 
which could be a cohesive device itself, located in the preceding sentence. Finally, a remote tie 
is one in which the cohesive device is related to the presupposed item without the assistance of 
intermediate ties. 

The model proposed by Halliday and Hassan may be described as follows. This model is 

developed to analyze but not quantify cohesion. The authors' taxonomy, however, does not 
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provide for a way to predict whether the presence of cohesive devices within a given text will 
result in cohesive discourse. One characterization of the cohesion was assigned to each text/ A 
measure of the density of cohesive devices in each major category was obtained by dividing the 
number of cohesive devices belonging to one category, e.g. reference, by the number of relevant 
opportunities of occurrence. In the case of the cohesive category of reference the relevant 
opportunity of occurrence is noun phrases. The measure of cohesiveness was established 
according to the procedures described hereafter. The categories analyzed were reference, lexical 
and conjunction. Ellipsis and substitution will not be discussed because of the low density of 
tokens. 

At first thought one would think that we could establish the density or frequency of 
cohesive devices in a given text by dividing the total number of cohesive devices by the number 
of sentences, clauses or words present. However, since each category of cohesive devices (RP, L 
and C) has different possibilities of occurrence we must establish the density of each particular 
category'. 

A cohesive tie consists of a cohesive item plus the presupposed item to which it refers. A 
cohesive tie between two clauses occurs when a conjunction links two clauses and establishes a 
relationship between them. Whenever we have two clauses it is possible to join them by using a 
conjunction. Therefore, the possible number of conjunctions in a text depends upon the number 
of clauses in the text. The following steps were taken to compile relevant data (Appendix #6): 

1 . The number of clauses, sentences, NPs and words and the cohesive types in each text 
were counted. 

2. The frequency of occurrence of each of the main categories of cohesive devices was 
established. The number of cohesive devices from each category was divided by the number of 
possible occurrences. For example, since conjunctions can appear between clauses, the number 
of conjunctions found in a given text was divided by the total number of clauses in this text. By 
the same token, since lexical cohesion can hold between any two words, the number oflexical 



cohesive ties was divided by the total number of words in each report. In the case of references 
the number of referential cohesive devices was divided by the number of NPs since pronouns, 
the major kind of referential device, need a presupposed noun phrase. 

The data was analyzed using the model proposed by Halliday and Hasan ( 1976). 
Modifications to the original model include the following: 

First, Halliday and Hasan use the sentence as their unit of analysis. In the interest of 
retaining more pertinent information I chose to use the clause as the unit of analysis. Other 
researchers dealing with cohesion, e.g. Almeida 1984, have chosen to use a T-unit for their 
analysis. 

Second. Halliday and Hasan's cohesion model deals with English produced by native 
speakers. In my work with nonnative speakers it was often not possible to make use of the 
precise language uttered by the students, but rather it became necessary to infer what the 
intended utterance was likely to have been. Also when counting words, clauses and NPs I chose 
to exclude false starts and unnecessary repetition. Unnecessary' repetition includes varieties 
where the speaker repeats a word or phrase in order to gain time. Also verbalized pauses such as 
c7?, ///?, even though they may appear in the transcript, were not included in the word count. 

Third, Halliday and Hasan include distance of cohesive items in their analysis. Such a 
measure was not relevant to my study since I did not have an acceptable way to quantify the role 
of distance in assessing cohesiveness of the texts. The number of cohesive devices from each 
major category (reference, lexical and conjunction) was converted into a frequency and 
correlated with the average grade for cohesion given by the three raters. 

While this model was designed to analyze cohesion in written discourse. I chose to use it 
to analyze spoken language. 1 believe that there is an overlap between the way cohesion in 
written discourse is created and the way cohesion in spoken discourse is created. However, there 
are differences between written and spoken discourses and, therefore, in the future a model of 
cohesion analysis of spoken text may need to be developed. The following are some of the 
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differences between written and spoken discourse which may pose a problem in analyzing 
cohesion. When judging the cohesion of a written text, decifering the written or typed word is 
seldom a problem. On the other hand, when judging the cohesion of a spoken discourse 
inaccurate pronunciation may interfere and distract us from focusing on cohesion and, therefore, 
our assessment may not be accurate. Long pauses in spoken language may also be a problem. If 
the pauses are too long then the rater may have difficulty in establishing a connection between 
the current sentence and the preceding sentence. If there is a long gap in the conversation raters 
may not know to whom a pronoun refers. In written text, long time gaps between sentence 
creation are irrelevant because ultimately the text appears contiguous on the page. Also in 
written discourse, mediate and remote cohesive ties are easy to establish because we can 
visualize the entire text. In spoken discourse, however, we cannot remember or visualize the 
entire text. Therefore, we may miss some less obvious cohesive ties. This can be circumvented 
by providing the rater with the transcript of the text. However, this is not advisable because 
when rating spoken language the rater should be allowed to listen to the tape only. Finally, 
another shortcoming of the model is that it does not take into account paralinguistic features 
such as intonation, pitch, etc. It can be the case that such features may be employed to mark 
cohesion in spoken language. This is something that researchers may want to look into further. 

The next step was to correlate the grade assigned for cohesion (based on the analysis of 
cohesion) to the grade of cohesion assigned by the three raters who listened to the tapes and 
graded cohesion according to a scale. The overall grade for cohesion is the axerage of the three 
grades given by three different experienced raters. 
Results and Discussion 

Given the small amount of data, I decided to check the degree of interdependence by 
comparing each of the three categories of cohesion with the overall grade for oral proficiency 
using Spearman's correlation coefficient. Using the same test the degree of interdependence 
among the three major categories of cohesive devices was established. Vor an aK).5 the only 
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significant correlation was between the number of referential items and the overall grade for 
cohesion. A similar test with Pearson's correlation coefficient yielded the same results. 
(Appendix #7) This implies that an increased number of referential ties could result in increased 
cohesive discourse. However, the results of the study were not conclusive. The indications 
yielded by the analysis and the statistics are not totally satisfactory. It could be the case that 
analyzing cohesion in spoken language may require a complementary model which will be more 
sensitive to features of spoken language, such as intonation and pitch. Also the measurement of 
cohesion should focus on the appropriateness of cohesive devices to the context and their 
distribution throughout the text. 

These preliminary findings indicate that the use of pronouns may be of some importance 
in creating a cohesive text. Since the only cohesive device that correlates with the grade given 
for cohesion is the referential pronouns, this may be an indication that the raters judgement 
about the cohesion of a text is influenced by the frequency of pronouns. If further study reveals 
similar results, the findings of this, along with the results of other relevant studies, could be 
including in the training for test raters. Test raters can be informed that paying attention to the 
nature and frequency of occurrence of cohesive devices may be a way to accurately make 
judgements about the cohesiveness of a text. Along the same lines, EFL instructors can share 
this finding with their students. Students can be informed that judgement of a test rater about the 
cohesiveness of a given text seems to be at least partially subject to the nature and frequency of 
occurrence of referential cohesive devices. 
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Appendix #2 



TEST FORMAT 

A. Group Discussion 

B. Oral Report 

C. Oral Interview 

D. Role Play 



GROUP DISCUSSION: 2-B 

You and your friends have been asked to 
give a report on tourism in Cyprus to a group of 
visiting American high school students. Time is 
short so decide what is the most: important 
information to talk about. Your friends may 
propose different information as more important 
than what you propose. Be ready to express your 
opinion, agree or disagree with your friends, 
and explain to them how your suggestions are 
better than theirs. 

Here is what you think you should talk 
about in the report: 

- The beach resorts of. Ayia Napa and. 
Protaras 

- beautiful beaches 

- big hotels 

- many sea sports 

- many bars and restaurants 
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SOBJECT: Male/Urban 
Group 4, Student A 

Appendix #5 
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Appendix #7 : Correlations between frequencies of cohesive devices and 
overall grade for cohesion 

Spearman Correlation Coefficient 

MATRIX OF SPEARMAN CORRELATION COEFFICIENTS 

REFE LEXI CONJ Overall 

Cohesion 



REFE 


1.000 






LEXI 


0.214 


1.000 




CONJ 


-0.039 


•0.172 


1.000 


Overall 








Cohesion 


0.573 


-0.088 


0.008 



Number of Observations: 16 



PEARSON CORRELATION MATRIX 

REFE LEXI CONJ Overall 

Cohesion 

REFE 1.000 

LEXI 0.113 1.000 

CONJ -0.065 -0.050 1.000 

Overall 

Cohesion 0.552 -0.153 0.033 1.000 



Number of Observations: 16 
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